Frequency Sensitive Hebbian Learning - Neural Networks, 1996., IEEE International Conference on

,4bstract: A new learning algorithm is proposed for the training of single layer linear networks. The network studied has an input layer of N units and an output layer of M units. The input and output layer are fully connected via a n M x N weighit matrix. It is well known that such a network of linear processing units will generate M principal components of the input distribution when it is trained by Hebbian type learning algorithms, such as General Hebbian Algorithm (GHA). It is also known that the same network structure of winner-take-all (WTA) units will produce M cluster centres of the input space when it is trained by gradient based competitive learning, such as Kohonen learning. The new algorithm also uses a Hebbian type learning, mechanism, but unlike the previous algorithms siuch as GHA, it simultaneously classifies the input distribution into M subclasses and extracts the principal component of each subclass distribution. To achieve robust performance, a frequency sensitive competitive learning mechanism is incorporated into the process, hence the new algorithm is called frequency sensitive Hebbian learning (FSHL). We have applied the new algorithm to image d,ata compression applications and simulation results are presented which indicate that the new FSHL will consistently outperform the optimal Karhunen-Loeve transform (KLT) and is competitive to Kohonen networks.